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LONG  TERM  GOALS 

The  goal  of  this  project  is  to  develop  methods  for  automated  analysis  of  zooplankton 
size  and  taxonomic  composition  using  images  from  theVideo  Plankton  Recorder  (VPR). 
The  goal  is  to  sort  plankton  to  species  or  genus  level  and  measure  body  size  in  real  time 
at  sea.  The  VPR  provides  sharply  focussed  video  images  of  plankton  and  seston  in  the 
size  range  100  microns  to  5  cm  and  has  an  on-board  CTD,  fluorometer,  flowmeter,  and 
transmissometer.  Currently  the  VPR  is  towed  at  0. 5-5.0  m/s  and  has  been  deployed  in 
shelf  and  oceanic  waters,  including  extensive  survey  work  on  Georges  Bank.  The  VPR 
has  also  been  deployed  on  the  ROV  JASON  for  individual  particle  tracking,  and  a 
moored  profiling  system  is  under  development,  which  will  include  on-board  image 
processing  and  satellite  telemetry  of  processed  data. 

OBJECTIVES 

Our  four  specific  objectives  are  to:  1)  use  our  existing  hardware  to  develop  methods  for 
real-time  detection  (60  fields  per  second)  of  in-focus  organisms  in  the  video  and  for 
storing  the  images  to  disk;  2)  develop  pattern-recognition  software  for  classification  of 
organisms  into  major  taxonomic  groupings  (copepods,  chaetognaths,  doliolids,  etc.);  3) 
develop  specifications  for  transferring  the  software  routines  to  hardware  to  enable  real- 
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time  sorting  of  zooplankton  according  to  major  taxa;  and  4)  to  develop  a  data-analysis 
and  display  system  for  real-time  visualization  of  plankton  distributions.  Development  of 
the  complete  image  processing/data  analysis  package  enables  incorporation  into  the 
VPR  system  for  real-time  identification  of  planktonic  taxa  and  visualization  of  their 
distributions  while  at  sea. 

APPROACH 

Our  approach  for  the  first  objective  was  to  develop  software  for  accurate  focus  detection 
and  then  transfer  it  to  our  IT- 151  image  processor.  The  approach  for  the  second 
objective  was  to  develop  software  for  identification  and  sorting  organisms  to  major  taxa. 
The  third  objective  involved  establishing  specifications  for  hardware  that  operates  the 
identification  and  sorting  software  in  real-time.  The  fourth  objective  was  to  develop  a 
data  analysis  system  for  viewing  the  spatial  patterns  at  sea  in  real  time. 

WORK  COMPLETED 

We  have  achieved  all  objectives  of  this  project  and  have  used  the  prototype  system  at 
sea  to  identify  and  visualize  planktonic  taxa  in  real  time  (Davis  et  al,  in  draft).  This 
project  has  involved  development  of  a  real-time  focus  detection  system,  development  of 
algorithms  for  feature  extraction  and  identification  of  planktonic  taxa,  and  development 
of  data  visualization  software  for  display  of  plankton  and  physical  data. 

RESULTS 

Objective  1:  We  have  developed  accurate  focus-detection  algorithms  and  implemented 
them  in  hardware.  Initially  this  was  done  using  an  Imaging  Technology  Inc.  Model  151 
(ITI-151)  real-time  image  processor  interfaced  to  a  Sun  SPARCstation  20,  but  we  have 
upgraded  the  system  to  an  ITI  Model  150-40  running  on  a  Pentium  PC  to  take  advantage 
of  the  rapid  I/O  of  the  PCI  bus.  The  old  system  missed  in-focus  objects  and  would  skip 
frames  due  to  an  I/O  bottle  neck  in  the  Sparc  S-Bus.  The  new  PCI  based  system  is  much 
faster  and  is  capable  of  extracting  all  in-focus  images.  A  video  source  (live  camera  or 
tape  player)  sends  a  video  signal  and  time  code  to  the  ITI  150-40  and  to  a  separate  time- 
code  reader  connected  to  the  PC  serial  port.  Algorithms  were  developed  for  reading 
field- accurate  (60-Hz)  time  code  into  the  PC  and  for  synchronizing  the  ITI  150-40 
pipeline  processor  for  real-time  region-of-interest  (ROI)  selection.  In-focus  ROIs  then 
are  stored  on  disk  as  TIF  files.. 

Our  capability  for  real-time  focus  detection  allows  for  rapid  automated  culling  of  blank 
and  out-of-focus  images,  reducing  manual  video  processing  by  a  factor  of  more  than  50. 
We  have  used  the  focus-detection  system  coupled  to  a  point-and-click  user  interface  for 
rapid  analysis  of  zooplankton  distributions  at  sea  (Davis  et  al.,  1996).  We  used  this 
method  on  our  Georges  Bank  GLOBEC  cruises  to  observe  2-  and  3-D  plankton 
distributions  (Gallager  et  al.,  1996;  Benfield  et  al.,  1996;  Norrbin  et  al.,  1996;  Benfield 
et  al.,  submitted;  Ashjian  et  al.,  in  draft;  Davis  et  al.,  in  prep;  Gallager  et  al.,  in  prep) 


Objective  2:  In  addition  to  focus  detection,  we  also  have  developed  feature-extraction 
and  pattern-analysis  algorithms  for  classification  according  to  taxonomy  and  size.  Our 
new  algorithms  have  classification  accuracy  at  about  90%  for  several  plankton  taxa.  Our 
research  efforts  have  focused  on:  1)  gray-scale  morphology  and  granulometric  feature 
extraction;  2)  textural  feature  extraction;  and  3)  feature  orthogonality  and  optimal 
discrimination. 

1)  For  Binarization,  Dr.  Luc  Vincent  has  developed  a  new  approach  to  using  a  global- 
connectivity  histogram  of  the  image;  histogram  plateaus  indicate  which  threshold  values 
are  most  stable.  Granulometric  algorithms  were  improved,  especially  with  respect  to 
scale,  filtering,  and  normalization;  classification  results  indicate  good  discrimination  for 
plankton  images.  A  novel  "geodesic  radial  function,"  the  ordered  set  of  values  of  the 
propagation  function  along  the  boundary  of  the  binarized  plankton  mask  is  an  important 
feature.  The  function  captures  the  size  and  elongation  of  the  organism,  its  extremities 
and  their  sharpness,  and  overall  smoothness. 

2)  Postdoctoral  Investigator  Xiaoou  Tang  has  developed  a  recursive  segmentation 
scheme  inspired  by  unsupervised  clustering;  the  technique  offers  speed,  simplicity,  and 
performance  comparable  with  the  global-connectivity  histogram.  Next,  a  set  of 
geometrical  features  (e.g.,  area,  perimeter,  shape  factor,  rotation-  and  scale-invariant 
moments)  are  computed  from  the  binary  segmented  image.  Other  textural  features  (e.g., 
run  length,  co-occurrence,  gray-level  difference,  wavelet,  power  spectrum,  fractal 
dimension)  are  also  extracted  from  the  original  gray-scale  images.  These  and  other 
feature  vectors  were  developed,  including  the  geodesic  radial  function  and  a  boundary 
shape  descriptor,  and  incorporated  into  a  robust  feature  set. 

3)  With  very  large  feature  vectors  much  information  is  redundant,  demanding  great 
computational  power  and  also  showing  poor  results.  An  important  step  is  the 
decorrelation  and  feature  selection.  Tang  developed  a  new  dominant  principal 
component  analysis  approach  on  the  combined  large  feature  vectors  formed  by  various 
features  described  above.  Then  the  selected  dominate  features  are  sorted  according  to 
their  discrimination  power  by  the  Bhattacharyya  distance  measure.  Dramatic 
improvement  of  the  classification  accuracy  is  achieved  by  only  using  a  simple  statistical 
classifier. 

Objective  3:  Once  the  training  sets  and  related  software  were  established,  the  focus, 
sizing,  and  identification  algorithms  allow  completely  automated  analysis  of  VPR 
images.  We  have  used  the  feature  extraction  and  classification  algorithms  on  our 
workstations  to  achieve  real-time  processing  and  classification  of  the  VPR  images  as 
they  are  acquired  at  sea. 

IMPACT 


The  new  sampline  capability  developed  in  this  project  will  provide  new  insights  into  the 
processes  controlling  plankton  distributions  and  thus  sound  and  light  transmission  in  the 
sea.  The  ability  to  visualize  the  distribution  of  planktonic  taxa  (and  size  structure)  at  sea 


in  real  time  allow  for  a  better  understandin  of  pelagic  biology.  No  longer  will  we  be 
probing  in  the  dark,  only  able  to  analyze  the  point  source  data  once  samples  have  been 
processed  in  the  laboratory  months  or  years  later.  The  dynamical  nature  of  the  pelagic 
environment  necessitates  the  use  of  tools  which  can  provide  rapid  visualizations  of 
plankton  distributions  together  with  physical  properties  of  the  water.  The  prototype 
system  we  have  developed  provides  this  capability. 

TRANSITIONS 

The  real-time  sampling  system  can  be  used  in  any  studies  in  which  it  is  desirable  to 
measure  plankton  abundance  including  sound  and  light  scattering  studies  and  ecological 
studies  of  environmental  quality  and  fisheries  ecology. 

RELATED  PROJECTS 

The  video  processing  procedures  we  have  developed  are  closely  coupled  to  our 
NSF/NOAA  funded  research  on  Georges  Bank  (GLOBEC)  and  our  NSF-funded 
development  of  a  vertically  profiling  VPR. 
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